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Aim of this report 


To summarise the outputs and plans of a 3-day working meeting for the expert working group 
(WG) on acceptance and demand data, 1 to 3 May 2019 at WHO Headquarters in Geneva. 


Background to the May 2019 meeting 


The group’s objective is to advance in the development of tools and guidance to enable 
immunization programmes and partners to measure and address local reasons of under- 
vaccination, and to track consistent and comparable data over time at a national and global 
level. The standardised tools under development will include quantitative survey questions for 
caregivers of children and qualitative interview guides for caregivers and healthcare workers. 
There will also be a related user guide. These tools will support high-quality data collection, 
analysis and application of findings to immunization programmes. They will also be developed 
with a long-term view conversant with coming generational changes in decision influences. 


The WG was launched by World Health Organization in November 2018 as a workstream under 
the larger multi-partner Demand Hub and in consultation with UNICEF, the US Centers for 
Disease Control (CDC), Gavi, the Vaccine Alliance, and the Bill and Melinda Gates Foundation. 
Members of the group include representatives of global agencies and experts from multiple 
geographical regions, covering a range of social and behavioural science disciplines with practical 
experience in high, middle and low-income settings. 


Since the launch of this group, a series of monthly teleconferences have taken place to support 
information sharing, planning, and early work to review the literature and generate insights from 
a range of relevant projects within and outside of immunization. This background work served as 
a foundation for the 3-day in-person meeting in May 2019. 


May 2019 meeting objectives 


- Build a common understanding of working group objectives and planning, and strengthen 
collaboration. 

- Enhance familiarisation with diverse country needs, existing data collection mechanisms, 
and areas that WG outputs will support. 

- Learn from comparable global efforts in developing and implementing standardised data 
collection tools in areas of health outside of immunization. 

- Integrate findings from an initial rapid needs assessment based on end-user informant 
interviews from regions and countries. 

- Agree ona framework for the development of qualitative and quantitative tools, and 
related user guidance. 


Preparations and inputs 


In the lead-up to the meeting, the WG members carried out a range of preparations and 
considered methodological issues. This included literature reviews of evidence for the factors 
affecting vaccination uptake, and discussions with key stakeholders, experts, as well as regional 
and country staff from WHO and UNICEF. These activities occurred in addition to the monthly 
teleconferences of the WG and helped to build a common understanding of the needs and 
offered initial directions for the tool design. 


A summary of preparations that took place prior to the meeting: 


- Identified known determinants of vaccination from existing systematic reviews. 

- Identified relevant theories and frameworks that can help structure the tools. 

- Identified and reviewed existing self-report survey instruments. 

- Built a pool of constructs/domains and associated question items. 

- Identified user needs and capabilities using a series of pilot informant interviews with 
colleagues and partners at a regional and country level. 

- Decided on main question topic areas and developed an initial draft structure of the 
quantitative survey tool. 


Meeting outline 


Following initial introductions and a short framing presentation, Day 1 focused on revisiting the 
expected outputs of the WG and its guiding principles for tool development. The most important 
principles were agreed to be: 1) pragmatism — that the tools can be easily used and the results 
enable action; and 2) rigour — that the tools are robust, theory-driven, and predictive of 
behaviour. 


Meeting participants, from left to right: Kerrie Wiley, Neetu Abad, Gilla Shapiro, Alina Lack, 
Wenfeng Gong, Nick Sevdalis, Julie Leask, Monica Jain, Gustavo Correa, Noel Brewer, Saad Omer, 
Cornelia Betsch, Charles Wiysonge, Gillian SteelFisher, Lisa Menning, Eve Dubé. 


To provide an orientation on immunization programme data collection and reporting, the group 
explored multiple mechanisms including existing coverage surveys, Expanded Programme on 
Immunization (EPI) reviews, the WHO/UNICEF joint reporting forms, and case-based surveillance 
(see Annex 3). Opportunities were identified for harmonization so that where possible, the 
eventual tools can be integrated into existing processes where possible, with consideration of 
routine monitoring platforms as a possible long term solution. In addition, the WG considered 
methods for measuring global trends similar to the non-communicable disease risk factor 
surveillance surveys, the International Tobacco Control Survey and/or partnering with a global 
polling entity that conducts multi-country surveys. 


The group also heard the interim findings of pilot informant interviews carried out with regional 
and country focal points. These offered insights into need, anticipated use, and practical 
considerations. The day finished with an initial discussion regarding the structure and scope of 
what can be measured by the quantitative tool. It was concluded that the main users of the tools 
would be EPI managers, and implementing partners, e.g. UNICEF and WHO in-country staff. 


Day 2 focused on preliminary steps for developing the qualitative and quantitative tools. 
Quantitative tool 


The quantitative tool is for use with caregivers. The findings would enable insights into 
behavioural and social drivers of vaccine uptake among children aged under 5 years. This 
incorporates those arising from the experiences of caregivers. To align the quantitative tool with 
the factors affecting uptake, the group carefully considered frameworks and models that guide 
action. The main considerations were the COM-B model’, Brewer et al.’s Increasing Vaccination 
Model? and a yet-to-be-published “Caregiver Journey” Framework (Annex 2). These models 
enabled the group to identify important question areas for the quantitative tools. A priority was 
to structure questions in such a way as to facilitate action, based on a key finding of the early 
informant interviews. 


Based on the considerations of what drives vaccination behaviour, the group agreed to 
incorporate four question areas into the quantitative tool and drafted indicators and early 
question items for each: 


- Thinking and feeling 

- Social processes 

- Readiness to vaccinate 
- Practical factors 


The tool should eventually be tailored according to user needs such as having capacity for short 
and long formats. It would have standardisation through core recommended question items 
from each question grouping while also having sufficient flexibility to be adapted to diverse 
settings. 


1 Michie et al. (2011). The behaviour change wheel: A new method for characterising and designing behaviour change interventions. 
Implementation Science 6:42. doi: 10.1186/1748-5908-6-42 

2 Brewer et al. (2017). Increasing Vaccination: Putting Psychological Science into Action. Psychological Science in the Public Interest, 18(3), 149- 
207. doi.org/10.1177/1529100618760521 


Qualitative tool 


The qualitative tools are for use with caregivers, health care workers/community health workers, 
and immunization programme managers. A sub-group with relevant expertise considered how 
these tools would be structured, with the aims to have them harmonized to the survey tool. 


The qualitative tools could be used in four possible ways — as follows: 


- Pre-survey, as an exploratory approach to discover where to focus the subsequent 
quantitative survey; 

- Post-survey, to enrich and contextualise findings from the quantitative survey; 

- Listening for change, for longer term periodic use to identify previously unknown issues - 
“horizon scanning”; 

- Inthe context of emergencies, for rapid response with the view to offering understanding 
and informing responses to evolving public health situations. 


These four possible applications of the qualitative tools are not mutually exclusive and may be 
jointly applied (e.g., qualitative exploration, followed by quantitative survey, followed by 
qualitative post-survey elaboration of the survey findings). 


Day 3 began with some refinement of the qualitative and quantitative tools. The discussion then 
shifted into a planning phase, exploring key next steps for testing and evaluation of the tools and 
ongoing stakeholder involvement. 


The following areas were discussed at length: 


Project planning for the quantitative tools involves phases of question refinement. 
Cognitive testing will happen in 6-7 countries, that represent a matrix of level of 
population remoteness, household income, language, and potential for impact. Testing 
will coincide opportunistically in countries undertaking programme activities such as EPI 
reviews or similar. For the qualitative tools, there will be a survey of end users and 
mapping of existing guidance documents specific to qualitative research to inform 
development of the user guide. 


- Stakeholder engagement. The WG will disseminate a regular update on its progress and 
establish a web page containing information on the tool development process and 
opportunities for stakeholders provide input or contribute. The WG will also continue the 
end-user interviews. 


- Name of the WG. The strengths and limitations of the term “Acceptance and Demand” 
were considered. Since the tools attempt to identify all factors influencing uptake, they 
will capture demand, supply, experiential, structural and internal factors that affect 
vaccination. Given the importance of how language shapes understanding, assumptions 
and solutions regarding uptake, it was agreed to adopt an interim name “Measuring 
Behavioural and Social Drivers (BeSD) of Vaccination” working group. 


Meeting outputs 


In summary, the deliverables of the meeting were as follows: 


- Key insights from relevant studies completed to date and current programme data 
collection processes. 


- Agreement on guiding principles, scope and structure for the development of qualitative 
and quantitative tools, including an interim working model. Agreed on four areas for the 
quantitative tool: thinking and feeling, social processes, readiness to vaccinate, and 
practical issues. 

- Agreement on indicators and initial questions/items for the qualitative and quantitative 
tools (structured around the four main question areas). 

- Drafted an outline of the user guidance for each of the quantitative and qualitative tools. 

- Development of a plan for the development of the tools, including stages of cognitive and 
psychometric validation testing, as well as stakeholder consultations throughout the 
process. 

- Development of an outline for the user guidance, defining expected user groups and key 
content areas, with the aim of supporting local adaptation and implementation 

- Agreement on methods and activities to support the tool’s dissemination and use, 
primarily centred on a digital platform, data repository and analytic support. 


The WG will next continue drafting quantitative and qualitative tools, then over the following 
months will begin testing and refining items, and developing user guides with a view to 
disseminating final draft package of tools and guidance by mid-2020. 
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Annex 2: Models used to inform tool development 


The two models shown here were carefully considered as a basis for identifying important 
question areas for the quantitative tools. It was recognised that these models may also evolve as 
the work on tool development and testing advances. 


Increasing Vaccination Model 


What people 
think and feel 
Perceived risk, worry 
Confidence, trust 
Safety concerns 


Social processes 
Provider recommendation 
Social norms 
Gender norms and equity 
Sharing info, rumors 


Practical issues 
Vaccine availability 
Convenience, costs 


Service quality and satisfaction 


Requirements, incentives 
Intervention fatigue 


Vaccination 
Schedule appt 
Consent 
Accept vaccine 
Delay 
Refusal 


(Based on Brewer, Chapman, Rothman, Leask, and Kempe (2017), in Psychological Science for the Public Interest 
https://www.ncbi.nim.nih.gov/pubmed/29611455) 


Caregiver Journey Framework 


Basic Vx awareness 
+ Unaware of (need for) vaccination 


Practical knowledge 

+ Unaware of schedule / need for a 
specific vaccine (e.g., measles) 

+ Place and/or time of immunization 
unknown 

+ Unaware of need to return for 
subsequent doses (e.g., 2" or 3" 
dose of a particular antigen) 


Access to information 
+ Lack of information (generic) 


© Bill & Melinda Gates Foundation 


Confidence 

+ Caregiver does not believe in vaccination 
+ Fears about side effects / AEFI / safety 

+ Fear of needles / injection / pain 


Norms/social context 

* Cultural/religious beliefs against 
vaccination 

+ Rumors or misinformation 


Complacency 

+ Lack of motivation (generic) 

+ Vaccination not seen as important or 
necessary 


Agency/self-efficacy 
+ Family member (other than mother) 
did not allow 
+ No female vaccinator 


Trust (not Vx-specific) 
+ Vaccinators unfriendly / poorly trained / 
not trusted 


Negative past experience 
+ Negative past experience (generic) 


Child/caregiver unavailable 

* Child away / not available (e.g., 
traveling, absent from home or 
school during vaccination visit) 

+ Migration 

+ Family problem, including illness 


Complacency/delay 
+ Postponed until another time 
* Forgot to go 


Contraindications 
* Child ill — not brought 


Convenience 

* Too busy / competing priorities 
+ Caregiver working 

* Inconvenient timing 

+ Place of immunization too far 
+ Long waiting time 

* Security concerns 

Cost 


+ Cost (e.g., charge money to take 
vaccine, transport too expensive) 


Service availability 

+ Vaccinator not available 

+ Vaccinator too busy 

* Vaccine not available 
(e.g., stockout) 


Contraindications 
* Child ill — brought but not 
vaccinated 


Other missed opportunities 

+ Missed opportunity (generic) 

* Card not available 

+ Not enough children to open a vial 


Annex 3: Summary of programme data collection processes 


z Routine Monitoring EY Periodic Surveys 
r r 


Facility reports Population QS A Facility Integrated 
Administered doses based U ua Y based surveillance 
Vaccines used Vaccination status, KAP — Knowledge, * Numbers of VPD’s 
Sessions planned/held mortality Attitudes, Practices 
Comms activities MOV — Missed Opp. Case based reports 
Cold chain status DHS - Demographic for Vaccination * for Polio, Measles, 
AEFI’s and Health Survey EVM — Effective Rubella, serious 
MICS - Multiple Vaccine Management AEFI, other diseases 
Electronic systems: Indicator Cluster SARA — Service as relevant 
Health Management Survey Availability, Readiness Identifiers and 
Information Systems EPI Coverage TIP — Tailoring location 
Immunization survey Immunization Outcome, including 
Registries LOA’s — Lot Quality Programmes lab confirmation 
Logistics Man. systems Assessment EPI reviews ... Immunization status 
Other health programmes Other government programmes National Bureau of Statistics GIS Social media 


